System for monitoring event related data

ABSTRACT

A system for monitoring event related data including a sensor data analyzer, an event analyzer and an actuator is disclosed. The sensor data analyzer detects events based on sensor data, the event analyzer couples to the sensor data analyzer and estimates the size of the detected entire events based on event related data of the detected events from the sensor data analyzer, and the actuator couples to the sensor data analyzer and the event analyzer and actuates a predetermined device based on the estimated size of the detected entire events.

The present application is a Continuation application of Ser. No.15/546,717 filed on Jul. 27, 2017, which is a National Stage Entry ofPCT/JP2016/000783 filed on Feb. 16, 2016, which claims priority fromSingapore Patent Application 10201501222X filed on Feb. 17, 2015, thecontents of all of which are incorporated herein by reference, in theirentirety.

TECHNICAL FIELD

The invention relates to a surveillance system. More specifically, theinvention relates to a system for monitoring event related data.

BACKGROUND ART

Surveillance systems such as video surveillance systems have been usedin several areas for routine surveillance and for event detection.Abnormal events such as terrorism, riot, theft, fighting, fire, caraccidents, etc. are meant to be captured by such surveillance systems.

However, in some situations, a surveillance camera cannot captureabnormal events within the surveillance area at an appropriate size toadequately cover the abnormal event. Therefore, users, includingsecurity officers and police officers, are not able to understand anincident appropriately because a surveillance camera captures only apart of the whole of the event.

Thus, what is needed and an object of the disclosure is to provide animproved surveillance system that captures abnormal events within asurveillance area at a size appropriate to the abnormal event.Furthermore, other desirable features and characteristics will becomeapparent from the subsequent detailed description and the appendedclaims, taken in conjunction with the accompanying drawings and thisbackground of the disclosure.

SUMMARY OF INVENTION

In a first aspect of the present invention, a system for monitoringevent related data including a sensor data analyzer, an event analyzerand an actuator is disclosed. The sensor data analyzer detects eventsbased on sensor data. The event analyzer is coupled to the sensor dataanalyzer and estimates the size of the entire detected events based onevent related data of the detected events from the sensor data analyzer.The actuator is coupled to the sensor data analyzer and the eventanalyzer and actuates a predetermined device based on the estimated sizeof the entire detected events.

In a second aspect of the present invention, a method for monitoringevent related data is disclosed. The method includes detecting eventsbased on sensor data; estimating the size of the entire detected eventsbased on event related data of the detected events; and actuating apredetermined device based on the estimated size of the entire detectedevents.

In addition, the object is also achieved by a computer program thatachieves the method for monitoring event related data having theabove-described configurations with a computer, and a computer-readablerecording medium that stores the computer program.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying figures, where like reference numerals refer toidentical or functionally similar elements throughout the separate viewsand which together with the detailed description below are incorporatedin and form part of the specification, serve to illustrate variousembodiments and to explain various principles and advantages inaccordance with the present embodiment.

FIG. 1

FIG. 1 depicts a block diagram of a video surveillance system inaccordance with a first exemplary embodiment.

FIG. 2

FIG. 2 depicts an example of imaging range adjustment based on sound(zoom-out) in accordance with the first exemplary embodiment.

FIG. 3

FIG. 3 illustrates an example of imaging range adjustment based on sound(zoom-in and controlling the direction of the camera) in accordance withthe first exemplary embodiment.

FIG. 4

FIG. 4 depicts an example of imaging range adjustment based on the typeof sound event (zooming-out) in accordance with the first exemplaryembodiment.

FIG. 5

FIG. 5 depicts an example of imaging range adjustment using an odorsensor and infrared sensor (zooming-out) in accordance with the firstexemplary embodiment.

FIG. 6

FIG. 6 depicts a flowchart of a method of monitoring event related datain accordance with the first exemplary embodiment.

FIG. 7

FIG. 7 depicts a display showing the estimated size of the detectedentire events on a map in accordance with the first exemplaryembodiment.

FIG. 8

FIG. 8 depicts a look-up table for alarm notification system inaccordance with the first exemplary embodiment.

FIG. 9

FIG. 9 depicts a block diagram of a system in accordance with a secondexemplary embodiment.

FIG. 10

FIG. 10 depicts a computer system for implementing a method ofmonitoring event related data in accordance with the first and thesecond exemplary embodiment.

Skilled artisans will appreciate that elements in the figures areillustrated for simplicity and clarity and have not necessarily beendepicted to scale. For example, the dimensions of some of the elementsin the simulation visualization or steps in the flowcharts may beexaggerated in respect to other elements to help improve understandingof the present embodiment.

DESCRIPTION OF EMBODIMENTS

The following detailed description is merely exemplary in nature and isnot intended to limit the invention or the application and uses of theinvention. Furthermore, there is no intention to be bound by any theorypresented in the preceding background of the invention or the followingdetailed description. It is the intent of the following embodiments topresent an improved method for monitoring event related data.

First Exemplary Embodiment

FIG. 1 depicts a block diagram of a video surveillance system 100 inaccordance with a present embodiment. The video surveillance system 100includes a sensor data analysis part 104, an event size estimation part108 and a camera control part 110. The sensor data analysis part 104receives sensor data 102 captured by sensing device(s), and detects anabnormal event with related information, including the detected time,within a surveillance area. The event size estimation part 108 iscoupled to the sensor data analysis part 104 and estimates the size ofthe entire detected event using a time series of the detected event andsensing device information, including the device location stored withina sensing device information database 106. The camera control part 110controls the surveillance camera (e.g. zooming-in, zooming-out, changingthe direction of the camera) based on the estimated size of the entiredetected event. Alternatively, a display 112 may show information basedon the estimated size of the entire detected event. Also, an alarmnotification system 114 may send alarm to a predetermined destinationbased on the estimated size of the entire detected event. In an example,the system 100 may include one or more of the following predetermineddevice: a camera control part 110, a display 112 and an alarmnotification system 114.

The event size estimation part 108 advantageously estimates theappropriate size of entire detected events. And then, an actuatoractuates a predetermined sensing device to facilitate understanding ofthe situation. In one example, the surveillance camera captures thewhole event and facilitates security officers to understand thesituation quickly. Alternatively, a display device shows the estimatedsize of entire detected events on a map to assist security officers tounderstand the situation quickly. Users may customize the predetermineddevice to actuate in a predetermined manner in response to detection ofpredetermined situations.

Also, the size estimation of entire detected events is based on the typeof events detected by the sensor data analysis part 104. For example, ifthe type of detected event is an explosion, the size of the entiredetected event is considered large. Thus, an actuator actuates apredetermined device taking into account the large size of the entiredetected event. Therefore, users, including security officers and policeofficers, are able to understand an incident appropriately since asurveillance camera or other devices are adjusted automatically based onthe size of the incident (abnormal event) to capture the whole of theevent. The present embodiment enables the users to deal with theincident quickly and properly. The users may customize the size of theevent corresponding to the type of event. The users are also able toinvestigate the incident using the appropriate video footage after theincident.

FIG. 2 depicts an imaging range adjustment based on sound (zoom-out) 200in accordance with a present embodiment. Timeline 202 shows 4 steps (Ato D) for the imaging range adjustment 200. At step A, Field of View 206of a camera and microphone array 208 is set at an initial imaging range204. At step B, the camera and microphone array 208 captures sounds ofbreaking glass 210 from a direction of arrival 212. At step C, thecamera and microphone array 208 further captures sounds of shouting 214from a direction of arrival 216. At step D, the event size estimationpart 108 estimates the size of the event by calculating both right andleft maximum angles for Field of View within the direction of the camerain the camera and microphone array 208 and within a predetermined timewidth. Then, the camera control part 110 controls the camera in thecamera and microphone array 208 by comparing the current setting of thecamera with the estimated size of the event consisting of the maximumangles and Field of View 206 of the camera and microphone array 208 tocause the camera and microphone array 208 to zoom-out to adjustedimaging range 218.

FIG. 3 depicts an imaging range adjustment based on sound (zoom-in andcontrolling the direction of the camera) 300. Timeline 302 shows threesteps (A to C) for the imaging range adjustment 300. At step A, a Fieldof View 306 of a camera and microphone array 308 is set at an initialadjustment range 304. At step B, the camera and microphone array 308captures sounds of fighting 310 from a direction of arrival 312. At stepC, the event size estimation part 108 estimates the size of the event bycalculating both right and left maximum angles for Field of View withinthe direction of the camera and microphone array 308 and within apredetermined time width. Then, the camera control part 110 controls thecamera and microphone array 308 by comparing the current setting of thecamera with the estimated size of the event consisting of the maximumangles to cause the camera and microphone array 308 to control thedirection and Field of View of the camera to rotate and zoom-in to anadjusted imaging range 314.

FIG. 4 depicts an imaging range adjustment based on the type of soundevent (zoom-out) 400. Timeline 402 shows three steps (A to C) for theimaging range adjustment 400. At step A, a Field of View 406 of a cameraand microphone array 408 is set at an initial adjustment range 404. Atstep B, the camera and microphone array 408 captures a sound of anexplosion 410 from a direction of arrival 412. At step C, the event sizeestimation part 108 estimates the size of the event by the type of event(e.g. when the sensor data analysis part 104 detects a large-scale eventlike the explosion 410, the event size estimation part 108 decides thesize of the event is “Large”). Then, the camera control part 110controls the camera of the camera and microphone array 408 in responseto the received size of the event. When the size is “Large”, the cameracontrol part 110 zooms out the camera to a predetermined setting (thewidest setting). Then, the camera control part 110 controls thedirection and Field of View of camera to zoom-out to adjusted imagingrange 414. When the sensor data analysis part 104 detects a small-scaleevent like the theft, the event size estimation part 108 can decide thesize of the event is “Small”. Also, when the sensor data analysis part104 detects a medium-scale event like the fire, the event sizeestimation part 108 can decide the size of the event is “Medium”.

FIG. 5 depicts an imaging range adjustment using an odor sensor andinfrared sensor (zoom-out) 500. Timeline 502 shows four steps (A to D)for the imaging range adjustment 500. At step A, a Field of View 508 ofa camera and microphone array 512 is set at an initial adjustment range510. The sensor data analysis part 104 may receive sensor data capturedby any sensing device, such as an odor sensor 504 and an infrared sensor506, to detect abnormal events.

At step B, an abnormal odor 514 is detected at the odor sensor 504within a detected time. At step C, an unusual crowd 516 is detected atthe infrared sensor 506. At step D, the event size estimation part 108estimates the size of the event by calculating both right and leftmaximum angles for Field of View within the direction of the camera andwithin a predetermined time width using the sensor's locationinformation. Then, the camera control part 110 controls the surveillancecamera by comparing the current setting of the camera with the estimatedsize of the event consisting of the maximum angles. The camera controlpart 110 then controls the camera and microphone array 512 to zoom-outto an adjusted imaging range 518.

FIG. 6 depicts a flowchart 600 of a method for monitoring event relateddata in accordance with a present embodiment. A sensor data analyzerdetects events based on sensor data (S602). Based on the event relateddata of the detected events, the size of the detected entire events isestimated (S604). Then, a predetermined device is actuated based on theestimated size of the detected entire events (S606).

FIG. 7 depicts a display 700 showing the size of the detected entireevents on a map in accordance with a present embodiment. In the presentembodiment, sounds of breaking glass 702 are detected by a sensor, suchas a microphone. Next, crowds 704 are detected by a sensor, such as aninfrared sensor. After that sounds of fighting 706 are detected by asensor such as a microphone. Based on these event related data, thegeographical size of the detected entire events 708 is estimated by theevent size estimation part 108. A display 700 may be actuated to showthe estimated size of the detected entire events on a map.

FIG. 8 depicts a look-up table 800 for alarm notification system 114 inaccordance with a present embodiment. In the present embodiment, anevent or a plurality of events are detected by one or more sensors.Based on the event related data from the sensors, the size of thedetected entire events is estimated by the event size estimation part.The actuator actuates an alarm notification system 114 so that the alarmnotification system 114 selects a notification destination based on theestimated size of the detected entire events. For example, if theestimated size of the detected entire events is ‘Small’, the alarmnotification system 114 refers to a look-up table 800 and selects‘Police’ as a notification destination. Then, the alarm notificationsystem 114 sends alarm to ‘Police’. Alternatively, if the estimated sizeof the detected entire events is ‘Medium’ or ‘Large’, the alarmnotification system 114 selects ‘Fire and Ambulance Service Division’ or‘Counter-Terrorism Action Group’ as a notification destination based onthe look-up table 800. Users may customize the notification destinationand size of the detected entire events in the look-up table 800.

Second Exemplary Embodiment

FIG. 9 depicts a block diagram of a system 850 in accordance with thesecond exemplary embodiment. The system 850 includes a sensor dataanalyzer 860, an event analyzer 870 and an actuator 880. The sensor dataanalyzer 860 detects events based on sensor data. The event analyzer 870is coupled to the sensor data analyzer 860 and estimates the size of thedetected entire events based on event related data of the detectedevents from the sensor data analyzer 860. The actuator 880 is coupled tothe sensor data analyzer 860 and the event analyzer 870 and actuates apredetermined device based on the estimated size of the detected entireevents.

The sensor data analyzer 860 includes the sensor data analysis part 104of the first exemplary embodiment. The event analyzer 870 and theactuator 880 include the event size estimation part 108 of the firstexemplary embodiment.

By the above-described configuration, according to the second exemplaryembodiment, the improved system that can capture abnormal events withina surveillance area at a size appropriate to the abnormal event can beprovided.

The method and system of the described exemplary embodiments can beimplemented on a computer system 900, schematically shown in FIG. 10. Itmay be implemented as software, such as a computer program beingexecuted within the computer system 900, and instructing the computersystem 900 to conduct the method of the example embodiments.

Some portions of the description which follows are explicitly orimplicitly presented in terms of algorithms and functional or symbolicrepresentations of operations on data within a computer memory. Thesealgorithmic descriptions and functional or symbolic representations arethe means used by those skilled in the data processing arts to conveymost effectively the substance of their work to others skilled in theart. An algorithm is here, and generally, conceived to be aself-consistent sequence of steps leading to a desired result. The stepsare those requiring physical manipulations of physical quantities, suchas electrical, magnetic or optical signals capable of being stored,transferred, combined, compared, and otherwise manipulated.

Unless specifically stated otherwise, and as apparent from thefollowing, it will be appreciated that throughout the presentspecification, discussions utilizing terms such as “scanning”,“calculating”, “determining”, “replacing”, “generating”, “initializing”,“outputting”, or the like, refer to the action and processes of acomputer system, or similar electronic device, that manipulates andtransforms data represented as physical quantities within the computersystem into other data similarly represented as physical quantitieswithin the computer system or other information storage, transmission ordisplay devices.

The present specification also discloses apparatus for performing theoperations of the methods. Such apparatus may be specially constructedfor the required purposes, or may comprise a general purpose computer orother device selectively activated or reconfigured by a computer programstored in the computer. The algorithms and displays presented herein arenot inherently related to any particular computer or other apparatus.Various general purpose machines may be used with programs in accordancewith the teachings herein. Alternatively, the construction of morespecialized apparatus to perform the required method steps may beappropriate. The structure of a conventional general purpose computerwill appear from the description below.

In addition, the present specification also implicitly discloses acomputer program, in that it would be apparent to the person skilled inthe art that the individual steps of the method described herein may beput into effect by computer code. The computer program is not intendedto be limited to any particular programming language and implementationthereof. It will be appreciated that a variety of programming languagesand coding thereof may be used to implement the teachings of thedisclosure contained herein. Moreover, the computer program is notintended to be limited to any particular control flow. There are manyother variants of the computer program, which can use different controlflows without departing from the spirit or scope of the invention.

Furthermore, one or more of the steps of the computer program may beperformed in parallel rather than sequentially. Such a computer programmay be stored on any computer readable medium. The computer readablemedium may include storage devices such as magnetic or optical disks,memory chips, or other storage devices suitable for interfacing with ageneral purpose computer. The computer readable medium may also includea hard-wired medium such as exemplified in the Internet system, orwireless medium such as exemplified in the GSM (Global System for Mobilecommunications) mobile telephone system. The computer program whenloaded and executed on such a general-purpose computer effectivelyresults in an apparatus that implements the steps of the preferredmethod.

The computer system 900 comprises a computer module 902, input modulessuch as a keyboard 904 and mouse 906 and a plurality of output devicessuch as a display 908, and printer 910.

The computer module 902 is connected to a computer network 912 via asuitable transceiver device 914, to enable access to e.g. the Internetor other network systems such as Local Area Network (LAN) or Wide AreaNetwork (WAN).

The computer module 902 in the example includes a processor 918, aRandom Access Memory (RAM) 920 and a Read Only Memory (ROM) 922. Thecomputer module 902 also includes a number of Input/Output (I/O)interfaces, for example I/O interface 924 to the display 908, and I/Ointerface 926 to the keyboard 904.

The components of the computer module 902 typically communicate via aninterconnected bus 928 and in a manner known to the person skilled inthe relevant art.

The application program is typically supplied to the user of thecomputer system 900 encoded on a data storage medium such as a CD(Compact-Disc)-ROM or flash memory carrier and read utilising acorresponding data storage medium drive of a data storage device 930.The application program is read and controlled in its execution by theprocessor 918. Intermediate storage of program data may be accomplishedusing RAM 920.

The processor 918 may execute a set of instructions that perform themethod claimed with respect to FIG. 6. The processor 918 is configuredto receive sensor data 102 captured by the sensing devices disclosedwith regard to FIG. 1. The sensor data 102 may be received by sensors950 that are located at e.g. a remote surveillance area. The processor918 is configured to analyze the sensor data 102 to detect event(s) andevent related data. Based on the event related data of the detectedevent(s), the processor 918 is configured to estimate the size of thedetected entire events. The processor 918 sends out actuation data 952to one or more actuators 954 so that the one or more actuators 954 canactuate one or more predetermined devices such as a camera, a displayand an alarm notification system. The one or more predetermined devicesare actuated based on the estimated size of the detected entire events.

While exemplary embodiments have been presented in the foregoingdetailed description of the invention, it should be appreciated that avast number of variations exist. For example, those skilled in the artwill realize from the teachings herein that the present technology mayalso be applied to any part in the other type of chemical sensors.

It should further be appreciated that the exemplary embodiments are onlyexamples, and are not intended to limit the scope, applicability,operation, or configuration of the invention in any way. Rather, theforegoing detailed description will provide those skilled in the artwith a convenient road map for implementing an exemplary embodiment ofthe invention. It is understood that various changes may be made in thefunction and arrangement of elements and method of operation describedin an exemplary embodiment without departing from the scope of theinvention as set forth in the appended claims.

A part or the whole of the foregoing exemplary embodiment and thevariations thereof may be defined as a supplementary note citedhereunder. However, the exemplary embodiment and the variations are notlimited to the following supplementary note.

(Supplementary Note 1)

A non-transitory computer readable storage medium storing the programthat causes a computer to perform:

a process that detects events based on sensor data by a sensor dataanalyzer;

a process that estimates the size of the detected entire events based onevent related data of the detected events; and

a process that actuates a predetermined device based on the estimatedsize of the detected entire events.

This application is based upon and claims the benefit of priority fromSingapore Patent Application No. 10201501222X filed on Feb. 17, 2015,the entire disclosure of which is incorporated herein.

INDUSTRIAL APPLICABILITY

The present invention can be applied to, for example, video surveillancesystems which are used in several areas for routine surveillance.

REFERENCE SIGNS LIST

-   100 Video surveillance system-   102 Sensor data-   104 Sensor data analysis part-   106 Sensing device information database-   108 Event size estimation part-   110 Camera control part-   112 Display-   114 Alarm notification system-   200 Imaging range adjustment (zoom-out)-   202 Timeline-   204 Initial imaging range-   206 Field of View-   208 Camera and microphone array-   210 Breaking glass-   212 Direction of arrival-   214 Shouting-   216 Direction of arrival-   218 Adjusted imaging range-   300 Imaging range adjustment based on sound (zoom-in and controlling    the direction of the camera)-   302 Timeline-   304 Initial adjustment range-   306 Field of View-   308 Camera and microphone array-   310 Fighting-   312 Direction of arrival-   314 Adjusted imaging range-   400 Imaging range adjustment based on the type of sound event    (zooming-out)-   402 Timeline-   404 Initial adjustment range-   406 Field of View-   408 Camera and microphone array-   410 Explosion-   412 Direction of arrival-   414 Adjusted imaging range-   500 Imaging range adjustment using an odor sensor and infrared    sensor (zooming-out)-   502 Timeline-   504 Odor sensor-   506 Infrared sensor-   508 Field of View-   510 Initial adjustment range-   512 Camera and microphone array-   514 Abnormal odor-   516 unusual crowd-   518 Adjusted imaging range-   600 Flowchart of the method for monitoring event related data-   602 Step of detecting events-   604 Step of estimating the size of the detected entire event-   606 Step of actuating predetermined device based on the estimated    size of event-   700 Display-   702 Breaking glass-   704 Crowd-   706 Fighting-   708 Size of the detected entire events-   800 Look-up table-   900 Computer system-   902 Computer module-   904 Keyboard-   906 Mouse-   908 Display-   910 Printer-   912 Computer network-   914 Suitable transceiver device-   918 Processor-   920 Random Access Memory (RAM)-   922 Read Only Memory (ROM)-   924 I/O interface to display-   926 I/O interface to keyboard-   928 Interconnected bus-   930 Data storage device-   950 Sensors-   952 Actuation data-   954 Actuators

1. A surveillance system comprising: an information processing systemcomprising at least one processor configured to: detect events by usingsensor data; and estimate an entire size of the detected events based onevent related data of the detected events; and an alarm notificationsystem comprising at least one processor configured to: select anotification destination from a plurality of predetermined departmentsbased on the estimated entire size of the detected events; and send analarm to the selected notification destination.
 2. The surveillancesystem according to claim 1, wherein the plurality of predetermineddepartments include a department of police.
 3. The surveillance systemaccording to claim 2, wherein the at least one processor of the alarmnotification system is further configured to select the department ofpolice as the notification destination when the estimated entire size ofthe detected events corresponds to a smallest level.
 4. A surveillancemethod comprising: detecting events by using sensor data; and estimatingan entire size of the detected events based on event related data of thedetected events; and selecting a notification destination from aplurality of predetermined departments based on the estimated entiresize of the detected events; and sending an alarm to the selectednotification destination.
 5. The surveillance method according to claim4, wherein the plurality of predetermined departments include adepartment of police.
 6. The surveillance method according to claim 5,further comprising selecting the department of police as thenotification destination when the estimated entire size of the detectedevents corresponds to a smallest level.
 7. A non-transitorycomputer-readable storage medium storing a program that causes acomputer to perform: detecting events by using sensor data; andestimating an entire size of the detected events based on event relateddata of the detected events; and selecting a notification destinationfrom a plurality of predetermined departments based on the estimatedentire size of the detected events; and sending an alarm to the selectednotification destination.
 8. The storage medium according to claim 7,wherein the plurality of predetermined departments include a departmentof police.
 9. The storage medium according to claim 8, wherein theprogram further causes the computer to perform selecting the departmentof police as the notification destination when the estimated entire sizeof the detected events corresponds to a smallest level.